Skip to content

Keep a new iOS request when a power loss undoes its entry save - #79

Open
dmurphy5 wants to merge 6 commits into
masterfrom
dylan/ios-entry-save-durability
Open

dmurphy5 wants to merge 6 commits into
masterfrom
dylan/ios-entry-save-durability

Conversation

@dmurphy5

@dmurphy5 dmurphy5 commented Nov 12, 2026 •

Copy link
Copy Markdown

What could go wrong

On iOS, each request lives in its own folder, with its details in entry.json. To save a file, the library writes <file>.tmp, flushes it to disk, and renames it into place. It did not flush the folder after the rename.

So after a power loss or a kernel panic right after mutate() resolved, the folder could hold only entry.json.tmp. Nothing reads that file. The request was gone, and the app still believed it was queued. The outcome journal had the same gap: a finished request's outcome file could be lost while its row survived. Android already flushes the folder after every rename.

What this PR does

  • Flush the folder after every atomic write. FileIO.writeAtomically now flushes the parent folder after the rename, as Android does. This covers entries, staged bodies, the outcome journal, the task map, settings and the import marker. An entry's first save also flushes the queue's root folder, which holds the new request folder's name. A failed flush is logged and the write still succeeds, so mutate() and a settle never fail because of it.
  • Recover a lone entry.json.tmp at startup. Before anything loads the queue, a folder with entry.json.tmp and no entry.json is checked. If the tmp is a complete entry for that folder, it becomes entry.json and runs like any other request. If its body file is gone, it settles error with errorKind: 'file', the same as any request with a missing body. A tmp that does not read as an entry is left on disk.
  • entry.json wins over a stray tmp. A tmp next to a readable entry.json is a later save that did not finish. The tmp is deleted.
  • A failed write deletes its tmp. Before, a save that failed at the flush or the rename left the tmp behind. Startup would then bring back a request whose mutate() had rejected.
  • Removing a request deletes its stray tmp first, and flushes that. Otherwise a removal cut short could leave a lone tmp, and startup would bring back a finished or cancelled request.

Why every write and not only the first entry save: without the flush, a power loss can also undo a later rename. For an entry, the older entry.json comes back and startup keeps it, which can undo a change that mutate() already confirmed. For the journal, a completed row could survive without its outcome file, and startup would forget the row before JS got the outcome. Review measured a folder flush on a Mac (not an iPhone) at about 0.01 ms with nothing pending, and about 0.7 ms right after an unsynced 20 MiB part write.

What this PR does not do

  • It uses plain fsync, as Android does. On Apple platforms, fsync hands the data to the drive but does not make the drive write its cache. F_FULLFSYNC does that but is much slower. Switching is left as a separate decision.
  • It does not flush the parent of the queue root, or the folder of a staged body that is written without writeAtomically (form and file copies, chunked moves). Those land in the request folder, which the entry save then flushes.
  • iOS has no diagnostic log on master, so recovery and flush failures are logged with NSLog, like other iOS storage problems.
  • A request whose mutate() never resolved, because the app died mid-call, is sent after the next start. If the app then creates the same thing again under a new id, the server receives it twice.

How it was tested

  • New EntrySaveDurabilityTests, 19 tests. They cover the flush after the rename for any atomic write and for each entry save, the flush after a failed write's tmp delete, the root flush on the first save, the journal flush before the completed entry save, a failing flush during enqueue and settle, the failed-write cleanup, a rejected enqueue staying gone, recovery and a normal run of a recovered request, a repeat mutate() on a recovered request, recovered simple and chunked requests with a missing body, a tmp that does not decode, a tmp in the wrong folder, a tmp next to a readable and an unreadable entry.json, removal order, and v9 and set-aside folders being left alone.
  • Against master with only the test seams added, 14 of the 19 fail. The other 5 check that something is left alone, or that the real folder flush works.
  • Moving the flush before the write makes 3 of them fail. Flushing only after a successful write makes the failed-write test fail.
  • swift test in ios/: 277 tests, 0 failures.
  • yarn lint:ci, yarn typecheck, and yarn test (199 tests) pass.
  • Android code is not changed, so its suite was not run.

🤖 Generated with Claude Code

dmurphy5 and others added 3 commits November 12, 2026 15:20
The entry save wrote entry.json.tmp, synced the file, and renamed it onto
entry.json, but never synced the directory. A power loss right after
mutate() resolved could leave only entry.json.tmp, which nothing reads, so
the request was lost without a trace.

- Every entry save now syncs the entry directory after the rename. The
  first save also syncs the queue root, which holds the new directory's
  name. A failed sync is logged and does not fail the save.
- At startup, before anything loads the entries, a lone entry.json.tmp that
  decodes to the entry of its directory becomes entry.json. A tmp that does
  not decode is left on disk. A tmp next to a readable entry.json is a later
  save that did not finish; entry.json wins and the tmp is deleted.
- A write that throws now deletes its tmp, so a rejected enqueue cannot come
  back at startup.
- Removing an entry deletes a stray entry.json.tmp first and syncs that, so
  a removal cut short cannot leave a lone tmp to bring back.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Review follow-up. The outcome journal renamed its file into place without
a directory sync, while the completed entry save after it was synced. A
power loss could then keep the completed row and lose the outcome file, and
boot would forget the row before JS saw the outcome.

- FileIO.writeAtomically now syncs the parent directory after the rename,
  as Android does. Entries, staged bodies, the journal, the task map,
  settings and the import marker all get it. QueueStore keeps the extra
  root sync on an entry's first save.
- A failed sync is still logged and never thrown. The test seam moves to
  FileIO.syncDirectory.
- Tests pin the sync after the rename and the journal sync before the
  completed entry save.
- The tmp-delete lines in writeAtomically match the open storage-cause
  branch, so both merge cleanly.
- CHANGELOG states what the code does. README says a failed sync does not
  fail the call, and that iOS fsync does not empty the drive cache.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…exactly

- New test: a write whose rename fails throws, deletes its tmp, and syncs
  the parent directory once, after the tmp is gone.
- CHANGELOG: only data bodies go through the atomic write. Form, file and
  chunked bodies are placed by a rename or move, and the entry save that
  follows syncs the same directory before mutate() resolves.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
dmurphy5 and others added 3 commits November 13, 2026 14:56
Brings in #65, #74, #78 and #80.

CHANGELOG.md: kept every entry; master's entries first, then this branch's
(entry save survives a power loss).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The isCopiedBodyFile comment from #78 said FileIO.writeAtomically does not
sync the directory. This branch makes it sync, and boot promotes a tmp
that reads as the directory's entry before the cleanup runs. The rule is
unchanged: a directory with entry.json.tmp is never deleted. The comment
now says a tmp still there is one boot promotion could not use.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Brings in #71 (mutate() waits, onWarning).

CHANGELOG.md: kept every entry; master's entries first, then this branch's
one.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant